Skip to content

Faster deleteAcsSnapshotsAfter #1070

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
rautenrieth-da wants to merge 1 commit into from
Closed

Faster deleteAcsSnapshotsAfter #1070

rautenrieth-da wants to merge 1 commit into from
+59 −6

Conversation

@rautenrieth-da
Copy link
Contributor

@rautenrieth-da rautenrieth-da commented Jun 11, 2025

Attempt to speed up deleting acs snapshots by:

  • Not running multiple transactional delete queries in parallel
  • Merging multiple delete queries into one

@rautenrieth-da
faster deleteAcsSnapshotsAfter
6ae8571 
[ci]

Signed-off-by: Robert Autenrieth <robert.autenrieth@digitalasset.com>
OriolMunoz-da
OriolMunoz-da approved these changes Jun 11, 2025
View reviewed changes
Copy link
Contributor

@OriolMunoz-da OriolMunoz-da left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the other PR already will fix the issue (seq scan over u_h_c) - but the changes in this PR are still good, thanks!

@rautenrieth-da
Copy link
Contributor Author

rautenrieth-da commented Jun 11, 2025

I think the #1069 already will fix the issue

We still need to speed up deleteAcsSnapshotsAfter. Store initialization is taking too long on CILR, which is blocking app startup:

INFO 2025-06-06T08:36:04.192453755Z [resource.labels.namespaceName: sv-6] [resource.labels.containerName: scan-app] This update history may be missing import updates, checking for corrupt ACS snapshots
INFO 2025-06-06T08:36:04.311723121Z [resource.labels.namespaceName: sv-6] [resource.labels.containerName: scan-app] Deleting ACS snapshots for history 1 with migration 1 and recordTime > 0001-01-01T00:00:00Z
INFO 2025-06-06T08:36:04.315337937Z [resource.labels.namespaceName: sv-6] [resource.labels.containerName: scan-app] Deleting ACS snapshots for history 1 with migration 2 and recordTime > 0001-01-01T00:00:00Z
INFO 2025-06-06T08:36:04.316469990Z [resource.labels.namespaceName: sv-6] [resource.labels.containerName: scan-app] Deleting ACS snapshots for history 1 with migration 3 and recordTime > 0001-01-01T00:00:00Z
INFO 2025-06-06T08:36:04.363233696Z [resource.labels.namespaceName: sv-6] [resource.labels.containerName: scan-app] Deleted 120 rows from acs_snapshot, expecting to delete 27189522 rows from acs_snapshot_data
INFO 2025-06-06T08:36:04.363955058Z [resource.labels.namespaceName: sv-6] [resource.labels.containerName: scan-app] Deleted 184 rows from acs_snapshot, expecting to delete 2331225 rows from acs_snapshot_data
INFO 2025-06-06T08:36:04.388617987Z [resource.labels.namespaceName: sv-6] [resource.labels.containerName: scan-app] Deleted 718 rows from acs_snapshot, expecting to delete 51890215 rows from acs_snapshot_data
INFO 2025-06-06T09:22:25.334215633Z [resource.labels.namespaceName: sv-6] [resource.labels.containerName: scan-app] Deleted 2331225 rows from acs_snapshot_data

The following simple test on a copy of CILR is still taking >1h to complete, so unfortunately selective removal of snapshots is not an option, we'll have to truncate both acs tables.

begin;
select min(first_row_id), max(last_row_id) from scan_sv_1.acs_snapshot where migration_id = 2;
delete from scan_sv_1.acs_snapshot where migration_id = 2;
delete from scan_sv_1.acs_snapshot_data where row_id >= 83470207 and row_id <= 113431985;
commit;

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@OriolMunoz-da OriolMunoz-da OriolMunoz-da approved these changes

@ray-roestenburg-da ray-roestenburg-da Awaiting requested review from ray-roestenburg-da

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@rautenrieth-da @OriolMunoz-da